IN THE CLAIMS: 



1 . (currently amended) A computer-implemented face recognition 
process for identifying a person depicted in an input image, comprising using a 
computer to perform the following process actions: 

creating a database of a plurality of model image characterizations, 
each of which represents the face of a known person that it is desired to identify 
in the input image as well as the person's face pose; 

training a neural network ensemble to identify a person and their 
face pose from a region which has been extracted from said input image and 
characterized in a manner similar to the plurality of model images, wherein the 
network ensemble comprises, a first stage having a plurality of classifiers each of 
which has input and output units and is dedicated to a particular pose range and 
outputs a measure of the similarity indicative of the similarity between said 
characterized input image region and each of said model image characterizations 
associated with the particular pose range of the classifier, and a fusing neural 
network as its second stage which combines the outputs of the classifiers to 
generate an output indicative of the person associated with the characterized 
input image region and the face pose of that person : and 

employing the network ensemble to identify the person associated 
with the characterized input image region and the face pose of that person . 



2-4. (cancelled) 



5. (previously presented) The process of Claim 1 , wherein the process 
action for training the neural network ensemble comprises an action of preparing 
each model image characterization from a model image depicting the face of a 
known person that it is desired to identify in the input image by, 

extracting the portion of the model image depicting said face, 
normalizing the extracted portion of the model image by resizing it 
to a prescribed scale if not already at the prescribed scale and adjusting the 
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region so that the eye locations of the depicted subject fall within a prescribed 
area, and 

cropping the extracted portion of the model image by eliminating 
unneeded portions of the image not specifically depicting part of the face of the 
subject to create a model face image. 

6. (previously presented) T he process of Claim 5, wherein the process 
action for training the neural network ensemble further comprises actions for; 

categorizing the model face images by assigning each to one of a 
set of pose ranges into which its associated face pose falls; 

for each pose range, 

choosing a prescribed number of the model face images of 
each person being modeled which have been assigned to the selected pose 
range, 

concatenating each of the chosen model face images to 

create a respective dimensional column vector (DCV) for each, 

computing a covariance matrix from the DCVs, 
calculating eigenvectors and corresponding eigenvalues 

from the covariance matrix, 

ranking the eigenvalues in descending order, 
identifying a prescribed number of the top eigenvalues, 
using the eigenvectors corresponding to the identified 

eigenvalues to form the rows of a basis vector matrix (BVM) for the pose range; 

and 

multiplying each DCV by each BVM to produce a set of 
principal components analysis (PCA) coefficient vectors for each model face 
image. 

7. (previously presented) The process of Claim 6, wherein the fusing 
neural network has at least enough output units to allow a different output to 
represent each person it is desired to identify at each of the pose ranges, and 
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wherein the process action of training the neural network ensemble further 
comprises the actions of: 

for each face recognition neural network, inputting, one at a time, 
each of the PCA coefficient vectors associated with the pose range of the face 
recognition neural network into the inputs of the network until the outputs of the 
network stabilize; 

initializing the fusing neural network for training; 

for each DCV, simultaneously inputting the PCA coefficient vectors 
generated from the DCV into the respective face recognition neural network 
associated the vector's particular pose range group until all the PCA coefficient 
vectors of every DCV have been input, and repeating until the outputs of the 
fusing neural network stabilize; and 

for each DCV, simultaneously inputting the PCA coefficient vectors 
generated from the DCV into the respective face recognition neural network 
associated the vector's particular pose range group and assigning the active 
output of the fusing neural network as corresponding to the particular person and 
pose associated with the model image used to create the set of PCA coefficient 
vectors. 

8. (previously presented) The process of Claim 7, wherein the process 
action of employing the neural network ensemble to identify the person depicted 
in the input image face region, comprises the actions of: 

preparing the face region extracted from an input image by 
normalizing and cropping the extracted regions, wherein said normalizing 
comprises resizing the extracted face region to the same prescribed scale if not 
already at the prescribed scale and adjusting the region so that the eye locations 
of the depicted subject fall within a prescribed area, and wherein the cropping 
comprises eliminated unneeded portions of the image not specifically depicting 
part of the face of the subject; 

concatenating the prepared face region to create a DCV; 

multiplying the DCV by each BVM to produce a set of PCA 
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coefficient vectors for the extracted face region; 

inputting each PC A coefficient vector in the set of PC A coefficient 
vectors into the respective face recognition neural network associated that 
vector's particular pose range group; and 

identifying the active unit of the output of the fusing neural network 
and designating the person and pose previously assigned to that unit as the 
person and pose associated with the extracted face region. 

9. (previously presented) T he process of Claim 1 further comprising 
process actions for: 

training the neural network ensemble to identify the person 
associated with the characterized input image region to be an unknown person if 
it does not match any of the model image characterization to a prescribed 
degree; and 

employing the neural network ensemble to identify the person 
associated with the characterized input image region to be an unknown person if 
it does not match any of the model image characterization to the prescribed 
degree. 

10. (currently amended) A face recognition system for identifying a 
person depicted in an input image, comprising: 

a general purpose computing device; and 
a computer program comprising program modules executable by 
the computing device, wherein the computing device is directed by the program 
modules of the computer program to, 

capture model images, each of which depicts at least one 
person of known identity, 

locate and extract regions within the model images, each of 
which depicts the face of a known person that it is desired to identify in the input 
image, 

determine the face pose for each of the face regions 



extracted from the model images, 

categorize each face region by assigning each to one of a 
set of pose ranges into which its associated face pose falls, 

train a neural network ensemble to identify a person and 
their face pose from a region that depicts the face of a person which has been 
extracted from said input image, wherein the network ensemble comprises, a first 
stage having a plurality of classifiers each of which has input and output units 
and is dedicated to a particular pose range and outputs a measure of the 
similarity indicative of the similarity between said input image region and each of 
said model image regions associated with the particular pose range of the 
classifier, and a fusing neural network as its second stage which combines the 
outputs of the classifiers to generate an output indicative of the person 
associated with the characterized input image region and the face pose of that 
person ; and 

employing the network ensemble to identify the person associated 
with the characterized input image region and their face pose. 

11-14. (cancelled) 

15. (previously presented) The system of Claim 10, wherein the sub- 
module for training the neural network ensemble comprises a sub-module for 
preparing each face region extracted from said model images by normalizing and 
cropping the extracted regions, wherein said normalizing comprises resizing 
each extracted face region to the same prescribed scale if not already at the 
prescribed scale and adjusting each region so that the eye locations of the 
depicted subject fall within the same prescribed area, and wherein said cropping 
comprises eliminating unneeded portions of the image not specifically depicting 
part of the face of the subject. 

16. (previously presented) The system of Clam 15, wherein the 
sub-module for training the neural network ensemble further comprises sub- 



modules for: 

(a) selecting a previously unselected one of the set of pose 

ranges; 

(b) choosing a prescribed number of the prepared face images 
of each person being modeled which have been assigned to the selected pose 
range; 

(c) concatenating each of the chosen prepared face images to 
create a respective dimensional column vector (DCV) for each; 

(d) computing a covariance matrix from the DCVs; 

(e) calculating eigenvectors and corresponding eigenvalues 
from the covariance matrix; 

(f) ranking the eigenvalues in descending order; 

(g) identifying a prescribed number of the top eigenvalues; 

(h) using the eigenvectors corresponding to the identified 
eigenvalues to form the rows of a basis vector matrix (BVM) for the selected 
pose range; 

(i) repeating actions (a) through (h) for each remaining pose 

range; 

(j) multiplying each DCV by each BVM to produce a set of 
principal components analysis (PCA) coefficient vectors for each face image. 

17. (previously presented) The system of Claim 16, wherein the fusing 
neural network has at least enough output units to allow a different output to 
represent each person it is desired to identify at each of the pose ranges, and 
wherein the sub-module for training the neural network ensemble further 
comprises sub-modules for: 

for each face recognition neural network, inputting, one at a time, 
each of the PCA coefficient vectors associated with the pose range of the face 
recognition neural network into the inputs of the network until the outputs of the 
network stabilize; 

initializing the fusing neural network for training; 



for each DCV f simultaneously inputting the PCA coefficient vectors 
generated from the DCV into the respective face recognition neural network 
associated the vector's particular pose range group until all the PCA coefficient 
vectors of every DCV have been input, and repeating until the outputs of the 
fusing neural network stabilize; and 

for each DCV, simultaneously inputting the PCA coefficient vectors 
generated from the DCV into the respective face recognition neural network 
associated the vector's particular pose range group and assigning the active 
output of the fusing neural network as corresponding to the particular person and 
pose associated with the model image used to create the set of PCA coefficient 
vectors. 

18. (original) The system of Claim 17, wherein the sub-module for 
employing the neural network ensemble to identify the person depicted in the 
input image face region and the pose associated with the face of the identified 
person, comprises sub-modules for: 

preparing the face region extracted from an input image by 
normalizing and cropping the extracted regions, wherein said normalizing 
comprises resizing the extracted face region to the same prescribed scale if not 
already at the prescribed scale and adjusting the region so that the eye locations 
of the depicted subject fall within a prescribed area, and wherein the cropping 
comprises eliminated unneeded portions of the image not specifically depicting 
part of the face of the subject; 

concatenating the prepared face region to create a DCV; 

multiplying the DCV by each BVM to produce a set of PCA 
coefficient vectors for the extracted face region; 

inputting each PCA coefficient vector in the set of PCA coefficient 
vectors into the respective face recognition neural network associated that 
vector's particular pose range group; and 

identifying the active unit of the output of the fusing neural network 
and designating the person and pose previously assigned to that unit as the 
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person and pose associated with the extracted face region. 

19. (previously presented) The system of Claim 10, further 
comprising sub-modules for: 

training the neural network ensemble to identify the person 
associated with the input image face region to be an unknown person if it does 
not match any of the face regions assigned to each pose range to a prescribed 
degree; and 

employing the neural network ensemble to identify the person 
associated with the input image face region to be an unknown person if it does 
not match any of the face regions assigned to each pose range to a prescribed 
degree. 

20. (currently amended) A computer-readable memory for use in 
identifying a person depicted in an input image, comprising: 

a computer-readable storage medium; and 
a computer program comprising program modules stored in the 
storage medium, wherein the storage medium is so configured by the computer 
program that it causes a computer to, 

input model images, each of which depicts at least one 
person of known identity, 

locate and extract regions within the model images, each of 
which depicts the face of a known person that it is desired to identify in the input 
image, 

determine the face pose for each of the face regions 
extracted from the model images, 

categorize each face region by assigning each to one of a 
set of pose ranges into which its associated face pose falls, 

train a neural network ensemble to identify a person and 
their face pose from a region depicting the face of a person which has been 
extracted from said input image, wherein the network ensemble comprises, a first 
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stage having a plurality of classifiers each of which has input and output units 
and is dedicated to a particular pose range and outputs a measure of the 
similarity indicative of the similarity between said input image region and each of 
said model image regions associated with the particular pose range of the 
classifier, and a fusing neural network as its second stage which combines the 
outputs of the classifiers to generate an output indicative of the person 
associated with the characterized input image region and the face pose of that 
person ; and 

employing the network ensemble to identify the person associated 
with the characterized input image region and their face pose. 

21-24. (cancelled) 

25. (previously presented) The computer-readable memory of Claim 20, 
wherein the sub-module for training the neural network ensemble comprises a 
sub-module for preparing each face region extracted from said model images by 
normalizing and cropping the extracted regions, wherein said normalizing 
comprises resizing each extracted face region to the same prescribed scale if not 
already at the prescribed scale and adjusting each region so that the eye 
locations of the depicted subject fall within the same prescribed area, and 
wherein said cropping comprises eliminating unneeded portions of the image not 
specifically depicting part of the face of the subject. 

26. (previously presented) The computer-readable memory of Clam 25, 
wherein the sub-module for training the neural network ensemble further 
comprises sub-modules for: 

(a) selecting a previously unselected one of the set of pose 

ranges; 

(b) choosing a prescribed number of the prepared face images 
of each person being modeled which have been assigned to the selected pose 
range; 
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(c) concatenating each of the chosen prepared face images to 
create a respective dimensional column vector (DCV) for each; 

(d) computing a covariance matrix from the DCVs; 

(e) calculating eigenvectors and corresponding eigenvalues 
from the covariance matrix; 

(f) ranking the eigenvalues in descending order; 

(g) identifying a prescribed number of the top eigenvalues; 

(h) using the eigenvectors corresponding to the identified 
eigenvalues to form the rows of a basis vector matrix (BVM) for the selected 
pose range; 

(i) repeating actions (a) through (h) for each remaining pose 

range. 

G) multiplying each DCV by each BVM to produce a set of 
principal components analysis (PCA) coefficient vectors for each face image. 

27. (previously presented) The computer-readable memory of Claim 26, 
wherein the fusing neural network has at least enough output units to allow a 
different output to represent each person it is desired to identify at each of the 
pose ranges, and wherein the sub-module for training the neural network 
ensemble further comprises sub-modules for: 

for each face recognition neural network, inputting, one at a time, 
each of the PCA coefficient vectors associated with the pose range of the face 
recognition neural network into the inputs of the network until the outputs of the 
network stabilize; 

initializing the fusing neural network for training; 

for each DCV, simultaneously inputting the PCA coefficient vectors 
generated from the DCV into the respective face recognition neural network 
associated the vector's particular pose range group until all the PCA coefficient 
vectors of every DCV have been input, and repeating until the outputs of the 
fusing neural network stabilize; and 

for each DCV, simultaneously inputting the PCA coefficient vectors 




generated from the DCV into the respective face recognition neural network 
associated the vector's particular pose range group and assigning the active 
output of the fusing neural network as corresponding to the particular person and 
pose associated with the model image used to create the set of PCA coefficient 
vectors. 

28. (original) The computer-readable memory of Claim 27, wherein the 
sub-module for employing the neural network ensemble to identify the person 
depicted in the input image face region and the pose associated with the face of 
the identified person, comprises sub-modules for: 

preparing the face region extracted from an input image by 
normalizing and cropping the extracted regions, wherein said normalizing 
comprises resizing the extracted face region to the same prescribed scale if not 
already at the prescribed scale and adjusting the region so that the eye locations 
of the depicted subject fall within a prescribed area, and wherein the cropping 
comprises eliminated unneeded portions of the image not specifically depicting 
part of the face of the subject; 

concatenating the prepared face region to create a DCV; 

multiplying the DCV by each BVM to produce a set of PCA 
coefficient vectors for the extracted face region; 

inputting each PCA coefficient vector in the set of PCA coefficient 
vectors into the respective face recognition neural network associated that 
vector's particular pose range group; and 

identifying the active unit of the output of the fusing neural network 
and designating the person and pose previously assigned to that unit as the 
person and pose associated with the extracted face region. 

29. (previously presented) The computer-readable memory of 
Claim 20, further comprising sub-modules for: 

training a neural network ensemble to identify the person 
associated with the input image face region to be an unknown person if it does 



12 




not match any of the face regions assigned to each pose range to a prescribed 
degree; and 

employing the neural network ensemble to identify the person 
associated with the input image face region to be an unknown person if it does 
not match any of the face regions assigned to each pose range to a prescribed 
degree. 

30. (cancelled) 

31 . (previously presented) The process of Claim 1 , wherein the process 
action for training the network ensemble comprises an action of deriving each 
model image characterization from a set of model images of people, wherein 
each model image of the same person shows that person at a different face 
pose, said deriving action comprising: 

extracting the portion of each model image depicting a face; 

normalizing the extracted portion of each model image by resizing it 
to a prescribed scale if not already at the prescribed scale and adjusting the 
region so that the eye locations of the depicted subject fall within a prescribed 
area; 

cropping the extracted portion of each model image by eliminating 
unneeded portions of the image not specifically depicting part of the face of the 
subject to create a model face image; 

concatenating each of the model face images to create a respective 
model dimensional column vector (DCV) for each, 

categorizing the model DCVs by assigning each to one of a set of 
pose ranges into which its associated face pose falls; 

inputting the model DCV of the each model face image falling in a 
particular pose range, one at a time, to a pre-selected classifier dedicated to the 
particular pose range. 

32. (previously presented) The process of Claim 31 , wherein the 
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process action of training the network ensemble further comprises the actions of: 
initializing the fusing neural network for training; 

simultaneously inputting the respective DCV of each model face 
image into all classifiers, until the DCV of every model image has been input, and 
repeating until the outputs of the neural network stabilize; and 

simultaneously inputting the respective DCV of each model face 
image into all classifiers, and assigning the active output the neural network as 
corresponding to the particular person and pose associated with the model 
image used to create the DCV. 

33. (previously presented) The process of Claim 32, wherein the 
process action of employing the network ensemble to identify the person 
depicted in the input image face region, comprises the actions of: 

preparing the face region extracted from an input image by 
normalizing and cropping the extracted regions, wherein said normalizing 
comprises resizing the extracted face region to the same prescribed scale if not 
already at the prescribed scale and adjusting the region so that the eye locations 
of the depicted subject fall within a prescribed area, and wherein the cropping 
comprises eliminated unneeded portions of the image not specifically depicting 
part of the face of the subject; 

concatenating the prepared face region to create a DCV; 

inputting the DCV of the face region into all classifiers; and 

identifying the active output of the neural network and designating 
the person previously assigned to that unit as the person associated with the 
extracted face region. 

34. (original) The process of Claim 33, further comprising a process 
action of specifying that the person designated as associated with the extracted 
face region has the face pose previously assigned to the identified active output. 

35-42. (withdrawn) 
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